Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: add config iavl-lazy-loading to enable lazy loading of iavl store #14189

Merged
merged 12 commits into from
Jan 31, 2023

Conversation

yihuang
Copy link
Collaborator

@yihuang yihuang commented Dec 7, 2022

Description

It improve start time of archive nodes (from dozens of minutes to seconds).

It seems to be running alright on production node, but not sure of unknown side effects.

related: cosmos/iavl#638


Author Checklist

All items are required. Please add a note to the item if the item is not applicable and
please add links to any relevant follow up issues.

I have...

  • included the correct type prefix in the PR title
  • added ! to the type prefix if API or client breaking change
  • targeted the correct branch (see PR Targeting)
  • provided a link to the relevant issue or specification
  • followed the guidelines for building modules
  • included the necessary unit and integration tests
  • added a changelog entry to CHANGELOG.md
  • included comments for documenting Go code
  • updated the relevant documentation or specification
  • reviewed "Files changed" and left comments if necessary
  • confirmed all CI checks have passed

Reviewers Checklist

All items are required. Please add a note if the item is not applicable and please add
your handle next to the items reviewed if you only reviewed selected items.

I have...

  • confirmed the correct type prefix in the PR title
  • confirmed ! in the type prefix if API or client breaking change
  • confirmed all author checklist items have been addressed
  • reviewed state machine logic
  • reviewed API design and naming
  • reviewed documentation is accurate
  • reviewed tests and test coverage
  • manually tested (if applicable)

@yihuang yihuang requested a review from a team as a code owner December 7, 2022 09:32
@yihuang yihuang marked this pull request as draft December 7, 2022 09:33
CHANGELOG.md Outdated Show resolved Hide resolved
@yihuang yihuang force-pushed the iavl-lazy-loading branch 2 times, most recently from d14d643 to cdcfc1d Compare December 7, 2022 09:46
yihuang added a commit to yihuang/cosmos-sdk that referenced this pull request Dec 7, 2022
…ackport cosmos#14189)

improve start time of archive nodes.

Update CHANGELOG.md
@yihuang yihuang changed the title feat: add iavl-lazy-loading to enable lazy loading of iavl store feat: add config iavl-lazy-loading to enable lazy loading of iavl store Dec 8, 2022
yihuang added a commit to yihuang/cosmos-sdk that referenced this pull request Dec 16, 2022
…ackport cosmos#14189)

improve start time of archive nodes.

Update CHANGELOG.md
@github-actions
Copy link
Contributor

github-actions bot commented Jan 8, 2023

This pull request has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

@github-actions github-actions bot added the stale label Jan 8, 2023
@yihuang
Copy link
Collaborator Author

yihuang commented Jan 8, 2023

unstall

@alexanderbez alexanderbez removed the stale label Jan 8, 2023
improve start time of archive nodes.

Update CHANGELOG.md

fix test
@tac0turtle
Copy link
Member

@yihuang want to update this to alpha.2?

@yihuang
Copy link
Collaborator Author

yihuang commented Jan 31, 2023

@yihuang want to update this to alpha.2?

done, added a bunch of replacements to make it build though, not sure what's the best way to do it, do I need to make a separate PR for store package?

@yihuang yihuang marked this pull request as ready for review January 31, 2023 00:56
@tac0turtle tac0turtle enabled auto-merge (squash) January 31, 2023 21:08
@tac0turtle tac0turtle added the A:automerge Automatically merge PR once all prerequisites pass. label Jan 31, 2023
@tac0turtle tac0turtle merged commit b266522 into cosmos:main Jan 31, 2023
@yihuang
Copy link
Collaborator Author

yihuang commented Feb 1, 2023

can we backport to 0.47 and 0.46, it's important UX improvement for existing chains. @tac0turtle

@tac0turtle
Copy link
Member

@Mergifyio backport release/v0.46.x

@tac0turtle
Copy link
Member

@Mergifyio backport release/v0.47.x

@tac0turtle
Copy link
Member

heads up these prs will need to be reduced to 0.19.5

mergify bot pushed a commit that referenced this pull request Feb 1, 2023
…tore (#14189)

Co-authored-by: Marko <marbar3778@yahoo.com>
(cherry picked from commit b266522)

# Conflicts:
#	CHANGELOG.md
#	cosmovisor/go.sum
#	go.mod
#	go.sum
#	server/util.go
#	simapp/go.mod
#	simapp/go.sum
#	store/go.mod
#	store/go.sum
#	tests/go.mod
#	tests/go.sum
#	x/nft/go.mod
#	x/upgrade/go.mod
#	x/upgrade/go.sum
@mergify
Copy link
Contributor

mergify bot commented Feb 1, 2023

backport release/v0.46.x

✅ Backports have been created

mergify bot pushed a commit that referenced this pull request Feb 1, 2023
…tore (#14189)

Co-authored-by: Marko <marbar3778@yahoo.com>
(cherry picked from commit b266522)

# Conflicts:
#	CHANGELOG.md
#	go.mod
#	go.sum
#	simapp/go.mod
#	simapp/go.sum
#	store/go.mod
#	store/go.sum
#	tests/go.mod
#	tests/go.sum
#	x/nft/go.mod
#	x/nft/go.sum
#	x/upgrade/go.mod
#	x/upgrade/go.sum
@mergify
Copy link
Contributor

mergify bot commented Feb 1, 2023

backport release/v0.47.x

✅ Backports have been created

yihuang added a commit to yihuang/cosmos-sdk that referenced this pull request Feb 1, 2023
…tore (cosmos#14189)

Co-authored-by: Marko <marbar3778@yahoo.com>
(cherry picked from commit b266522)

# Conflicts:
#	CHANGELOG.md
#	cosmovisor/go.sum
#	go.mod
#	go.sum
#	server/util.go
#	simapp/go.mod
#	simapp/go.sum
#	store/go.mod
#	store/go.sum
#	tests/go.mod
#	tests/go.sum
#	x/nft/go.mod
#	x/upgrade/go.mod
#	x/upgrade/go.sum
julienrbrt pushed a commit that referenced this pull request Feb 1, 2023
…tore (backport #14189) (#14863)

Co-authored-by: yihuang <huang@crypto.com>
Co-authored-by: marbar3778 <marbar3778@yahoo.com>
tac0turtle added a commit that referenced this pull request Feb 2, 2023
…tore (backport #14189) (#14862)

Co-authored-by: yihuang <huang@crypto.com>
Co-authored-by: marbar3778 <marbar3778@yahoo.com>
yihuang added a commit to yihuang/cosmos-sdk that referenced this pull request Feb 7, 2023
…tore (backport cosmos#14189) (cosmos#14862)

Co-authored-by: yihuang <huang@crypto.com>
Co-authored-by: marbar3778 <marbar3778@yahoo.com>
SpicyLemon added a commit to provenance-io/cosmos-sdk that referenced this pull request Feb 16, 2023
…518)

* fix: Fix Evidence CLI query flag parsing (backport cosmos#13458) (cosmos#14799)

Co-authored-by: Likhita Polavarapu <78951027+likhita-809@users.noreply.github.com>
Co-authored-by: HuangYi <huang@crypto.com>
Co-authored-by: Julien Robert <julien@rbrt.fr>

* fix: nested iterator on cache store (cosmos#14798)

* chore: update IAVL  (backport cosmos#14830) (cosmos#14835)

Co-authored-by: Marko <marbar3778@yahoo.com>

* Mark v0.46.8-pio-2 in the changelog and release notes.

* feat: add config `iavl-lazy-loading` to enable lazy loading of iavl store (backport cosmos#14189) (cosmos#14862)

Co-authored-by: yihuang <huang@crypto.com>
Co-authored-by: marbar3778 <marbar3778@yahoo.com>

* chore: bump IAVL to v0.19.5 (backport cosmos#14889) (cosmos#14892)

Co-authored-by: Julien Robert <julien@rbrt.fr>

* build(deps): bump tendermint to v0.34.25 (cosmos#14903)

* build(deps): bump tendermint to v0.34.26 (cosmos#14911)

* fix: return named error for write validators (backport cosmos#14919) (cosmos#14921)

Co-authored-by: mmsqe <mavis@crypto.com>
Co-authored-by: Julien Robert <julien@rbrt.fr>

* Bump Tendermint to v0.34.25 (from v0.34.24). (#498)

* Fix a few listener proto comments. (#499)

* build(deps): bump btcd in v0.46 (cosmos#14846)

Co-authored-by: yihuang <huang@crypto.com>
Co-authored-by: Julien Robert <julien@rbrt.fr>
Co-authored-by: Elias Naur <103319121+elias-orijtech@users.noreply.github.com>
Co-authored-by: Marko <marbar3778@yahoo.com>

* ci: update release slack channel and algolia api key (backport cosmos#14942) (cosmos#14945)

Co-authored-by: Julien Robert <julien@rbrt.fr>

* feat: support profiling block replay during abci handshake (backport cosmos#14953) (cosmos#14964)

Co-authored-by: yihuang <huang@crypto.com>
Co-authored-by: Julien Robert <julien@rbrt.fr>

* Revert "Revert cosmos#14349 - cache iterator optimizations (#444)"

This reverts commit cd9d192.

* Mark v0.46.8-pio-3 in the changelog and release notes.

* fix: avoid broadcasting tx through query (backport cosmos#15044) (cosmos#15047)

Co-authored-by: Facundo Medica <14063057+facundomedica@users.noreply.github.com>
Co-authored-by: Julien Robert <julien@rbrt.fr>

* Add CLI commands for sanction Tx gov props (#510)

* Fix the changelog.

---------

Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: Likhita Polavarapu <78951027+likhita-809@users.noreply.github.com>
Co-authored-by: HuangYi <huang@crypto.com>
Co-authored-by: Julien Robert <julien@rbrt.fr>
Co-authored-by: Marko <marbar3778@yahoo.com>
Co-authored-by: mmsqe <mavis@crypto.com>
Co-authored-by: Robert Zaremba <robert@zaremba.ch>
Co-authored-by: Elias Naur <103319121+elias-orijtech@users.noreply.github.com>
Co-authored-by: Facundo Medica <14063057+facundomedica@users.noreply.github.com>
ankurdotb pushed a commit to cheqd/cosmos-sdk that referenced this pull request Feb 28, 2023
…tore (backport cosmos#14189) (cosmos#14862)

Co-authored-by: yihuang <huang@crypto.com>
Co-authored-by: marbar3778 <marbar3778@yahoo.com>
jiujiteiro pushed a commit to realiotech/cosmos-sdk that referenced this pull request Mar 7, 2023
…tore (backport cosmos#14189) (cosmos#14862)

Co-authored-by: yihuang <huang@crypto.com>
Co-authored-by: marbar3778 <marbar3778@yahoo.com>
tsenart pushed a commit to meka-dev/cosmos-sdk that referenced this pull request Apr 12, 2023
larry0x pushed a commit to larry0x/cosmos-sdk that referenced this pull request May 22, 2023
…tore (cosmos#14189)

Co-authored-by: Marko <marbar3778@yahoo.com>
JeancarloBarrios pushed a commit to agoric-labs/cosmos-sdk that referenced this pull request Sep 28, 2024
…tore (backport cosmos#14189) (cosmos#14862)

Co-authored-by: yihuang <huang@crypto.com>
Co-authored-by: marbar3778 <marbar3778@yahoo.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
A:automerge Automatically merge PR once all prerequisites pass. C:Store C:x/nft C:x/upgrade
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants